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A workflow, a series of processes, and a software and hardware system to provide single-user 
control of all devices and systems in a live television production. 
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Switcher, Mixer, Automation, E-MEM, GPI, PVTV, VDCP, protocol, CORBA, 

The invention allows for a dramatic reduction of workforce in live television production while 
maintaining the ability of pre-production (newsroom) personnel to input and update content using 
their familiar tools and systems. Existing single-user production systems terminate that ability at 
an arbitrary time before the beginning of the broadcast. The continuous, dynamic updating of 
content as well as production control commands is a key innovation as is the means for attaining 
it. 
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ParkerVision's PVTV system was the first commercially viable offering of a single-user 
production control system, in this case one oriented towards television news programming. PVTV 
offers only a very limited interface with industry standard Newsroom Computer Systems (NRCS). 
This limits the utility of PVTV since long-standing NRCS technology has allowed NRCS systems 
to provide dynamic updating of scripts, story presentation order, production elements, and 
program timing to lake place seamlessly throughout the production cycle from initiation of 
planning of the day's program through the completion of the progress. 

PVTV interrupts this beneficial workflow in that it requires that the NRCS rundown and scripts be 
transferred to PVTV at some arbitrary time before the life broadcast begins. All subsequent 
changes to running order, production content, timing calculations, etc. must then be made within 
the PVTV system. This disables the primary tool used by newsroom personnel, the NRCS, 



without offering any alternative. In fact, except for the running order of the stories and the initial 
loading of scripts to the teleprompter, PVTV requires all production commands, sequences, and 
content references to be manually added to its time line. This constitutes a step backwards in 
terms of information flow. 

Figure 1 below illustrates how production information flows into a traditional "manual" 
production control room. 



Original Relationship Between Newsroom 
Computer System and Live Production System 




Figure 1 



Key points are that the NRCS remains a functional tool throughout the on-air presentation of the 
broadcast, and that the program producer's ability of manipulate the content of the broadcast is 
maintained.. Script entries as well as changes in the sequence of stories in the program are made in 
the NRCS at all times, not just prior to the beginning of the broadcast. Further, the inherent 
functionality in the NRCS to pass some production command and content information from the 
script documents to the production devices is maintained. An example is entry of character 
generator data into scripts. 

In this methodology, the majority of production commands are collated by a director or a person 
of similar skills and communicated either verbally or in the form of a text document to pertinent 
device operators. 

Figure 2 illustrates how existing single-user production systems assist in some areas of live 
program execution while actually regressing in terms of functionality in other areas. 

Primitive "Automated" Relationship Between 
Newsroom Computer System and Live Production 

System 




Newsroom Computer System 




Figure 2 



The first key point is that the NRCS ceases to be a useful tool except for script writing at an 
arbitrary time prior to the beginning of the live broadcast. 

• > 

At that arbitrary time, the program running order from the NRCS is transferred to the production 
system and all subsequent changes must take place within that production system. As in Figure I, 
production commands are collated by a director or person of similar skills, and then they are 
manually entered into the production system. The program producer must from that point forward 
verbally instruct they director of any changes, and the director must in turn manually edit the 
production system timeline to reflect those changes. 

As discussed in the Description of Invention below, these severe drawbacks can now be 
surmounted and additional functionality added through the employment of a dynamic exchange of 
data between the NRCS and the production system, elimination of the production system timeline 
as a controlling process, and addition of new technology that allows most if not all production 

commands and objects to be extracted from the NRCS without need of any human intervention. 



The invention is described in detail in Tsunami CRS Version 2.0 (Attachment 1), the entirety of 
which constitutes one practical embodiment of the invention. In the CRS and throughout this 
document, methods and technologies described in terms of news programming have equal 
applicability to other typrs of live television production 

It summary form, the invention is a combination of workflow and software and hardware in 
support of that workflow. The result of this combination is a system in which television 
production information is seamlessly, automatically, dynamically, and continuously ingested from 
an existing system designed specifically for the creation of television content and sequencing of 
that content for live presentation. Such an existing system is typified by the Newsroom Computer 
Systems offered commercially by the Associated Press, Avid/iNews, AutoCue and several others. 
Users of those systems create scripts for news stories, associate other production elements such as 
graphics, video server files, video tapes, camera shots, etc. with those stories, and sequence those 
stories and their associated production elements. 

The invention accesses all such information in the newsroom computer system (or other pre- 
production software system including commerical off-the-shelf software suites such as Microsoft 
Office) and stages the content for presentation as defined in the newsroom computer system 
sequence, allowing multiple disparate production devices to be controlled from a single point and 
by a single individual. 

The invention is an Integrated Newsroom Production system. In common with existing news 
production systems it is a control system which receives news rundowns from a NRCS (such as 
iNews and ENPS), and uses this as the starting point for controlling connected devices including 



video switchers, audio mixers, cameras and robotic pedestals, character generators, still stores, 
video and audio digital disk recorders, and VTRs. 

What is unique about this invention is the way it does this job. It converts the running order into 
an event list, which is an ordered sequence of N-MEMs. Tsunami does not employ timelines, 
where the items in the timeline have durations, nor does it employ macros, which are stored 
(learned) sequences of commands to be sent to the connected devices. 

The data in an N-MEM object is a snapshot of the overall state of the controlled devices. This 
extends the work (previously patented and trade marked by Grass Vailey) on developing E-MEM 
technology for switchers. Where an E^MEM (Effects Mem ory) is a snapshot of the state of a 
single device used to produce a desired artistic effect on the screen, the N-MEM is a snapshot of 
the overall state of all the controlled devices, used to produce an overall artistic effect. An N- 
MEM is a set of E-MEMs 

An example of such an effect, would be a TV anchor on the screen overlaid with a title on the 
lower part of the screen with his name on. To realize this example effect, the control system has 
to: 

• move the camera into position 

• take the camera source on the switcher background bus 

• recall the CG page 

• take the CG source on a switcher Key bus 

o raise the audio slider for the anchor(s) microphone 

and this is a very simple effect. 

The overall effect is called a style. The above style is a specific instance of an N-MEM which is 
close-up shot of an anchor with a title superimposed. A s contains one or more N-MEMs, each of 
which contains the superset of possibilities for realization of the style , For the above example, the 
Style would have 1 N-MEM, and this N-MEM could have an E-MEM for each of the possible 
anchors (e.g. anchor 1 (John), anchor 2 (Fred) , and achor3 (Jane3). In turn, each of the these 
anchors could be taken with either camera 1, 2, or 3. Thus the N-MEMs forming a style contain a 
range of legitimate possibilities. 

Styles are created using the Style Wizard, which guides the operator through the process of 
creating these styles. These styles form the television station's overall style, it's production 
'look". Each of the possible E-MEMs is learned into the style. Once the set of styles is created for 
a station, they may be used for years without further modification, or until it is decided to change 
the station's on-air look. 

The Producer of a news program does not need to know anything about individual E-MEMs, but 
can concentrate on associating styles with stones. Where there may be a bewildering array of 
possible E-MEMs and E-MEM combinations in a Style, there is a relatively small set of styles 
used in a particular show, generally less than 20. 

In the workflow of producing a news program, stories or slugs are entered into the rundown, in the 
order they will be shown. The producer associates a Tsunami Style with each story in this 
rundown by employing a software plug-in (TAP) to the newsroom computer system. The 
producer further starts the process of making specific selections from the set of style possibilities 
for the style, starting the process denoted as N-MEM parameterization. For example, for one 
story, the producer selects Ted (Anchor 2) as the presenter, for another he parameterizes the N- 



MEM by placing the text "Today in LA" as the title for a CG. Similarly he would select the clip 
on a video server for a SOT (sound-on-tape). The director the completes the process of 
parameterization, by making selections such as which camera to use, which VTR machine to use 
for a specific tape, which remote feed a live insert will come from, etc. 

A rules checker runs in the background and checks that each story element has an associated Style, 
and that it is fully parameterized. Hours before the show starts, the rules checker will indicate 
much is incomplete, when the show starts, ideally there should be no problems indicated. The 
rules checker also checks for resource conflicts, and local station policies.. 

The event list is the list of parameterized N-MEMS that have been associated with the stories in 
the running order. Playout of the event list is a manual operation. At its most basic, the user 
presses the *take' button. The current N-MEM is released, the next N-MEM goes on air, and the 
N-MEM after that is previewed. Before an N-MEM goes on air, it should be in the ready state - 
and the N-MEM is ready when all the controlled devices are cued. There is an E-MEM for each 
controlled device and each E-MEM is cycled through the states: idle, cueing, ready(cued), active 
(on air or on preview), and released. The N- MEM's state is a roll up of the E-MEM states, and it 
cycles through the states: idle, cueing, ready(cued), on-preview, on-air, and released. This is all 
described in much more detail in the (draft) Product Technical Specification. 

On aspect of great interest is that that although what goes on air is manually paced by pressing 
take, the cueing of devices is performed automatically. This is different from other systems which 
require the cueing of a device (such as a camera move) to be manually pre-programmed onto a 
timeline. When an E-MEM is released, Tsunami scans forwards through the event list for the next 
N-MEM which has an E-MEM for the same device, and brings this into the cueing state. 

The GUI shows the event list, and the state of each N-MEM, and the state of the E-MEMs. Of 
concern to the user during playout would E-MEM which are not cued, which would result in the 
overall N-MEM not being ready prior to use. 

* * 

The GUI and control panel both have more buttons than the 'take* described above. There are 
buttons to float (skip) an N-MEM, to bring on the audio in advance of video, to hold audio after 
the N-MEM is released, to roll tapes and DDRs, and more. All this too is described in the Product 
Technical Spec. 



Following an individual item through its production life: 

1 . A script is created in the NRCS 

2. The script is selected by a producer for inclusion in a given news broadcast 

. 3. The script is assigned a position in the sequence of stories (rundown) for that news broadcast 

4. The producer specifes the basic production treatment (sytle) of each story.. 

5. Various persons (writersj graphic artists, video editors, directors) add production detail to 
each story (N-MEM Parameterization) 

6. The rundown containing the story is executed on air. 

7. The N-MEMs associated with the story change the state(s) of production device when . 
triggered to do so by the human operator. 

The result of this story life-cycle is that the production information associated with each story is 
always current, and that it requires no intevention by the single operator of the Integrated 
Production System to remain current. 



Supplementary Material for N-MEM Inve ntion Disclosure- 



This invention is a means of making live television productions which are planned and 
assembled in pre-production as an ordered list of Styles. For production, these Styles are 
converted into an Effects List comprised of N-MEMs for the realization (play-out) of the 
show. 

This means of production is effective for television shows which are highly patterned 
and/or scripted such as sports events, quiz shows, talk shows, and news. This document 
describes the generalized means of production, and then describes an instance of its usage 
in the field of news production. 

A second invention is the grouping of an unordered set of Styles into a graphically or 
physically represented Style Box which can be randomly accessed. This is effective for 
the unordered or unscripted portions of a program such as chat sessions where the on-air 
performers can speak at will, or for example, the intervention of a scripted news 
production with unscripted live breaking news. 

The creator or producer of a television program conceives and puts together the show as a 
series of scene elements or Styles. This set of Styles forms the on-air look of the 
program. As such the Styles can be created weeks or months in advance of the day of live 
production. Each Style is a vocabulary element of the show which when put together into 
the show's Style List forms its overall narrative. All television consists of simultaneous 
video and audio generally emanating from a video switcher and audio mixer. So there is a 
video mix appearing on the screen accompanied by an audio mix. A Style is the 
description of a particular instance of this and the N-MEM is the means of producing it. 

Examples of Style are: 

• a single camera shot accompanied by a single microphone input 

• as above, with overlay graphics or titles from a CG 

• as above with one or more Video feeds overlaid in an inserted box produced by a 
DVE, and a more complicated audio mix 

• as above with an additional musical sound track played back from an audio disk 
recorder 

• video and audio played back from VTR or DDR 

• as above, with overlaid graphics 

• as above, with the recorded sound at background level, with live sound from a 
presenter in the foreground 

The variety and nature of the Styles are limited only by the creativity of the production 
staff and the overall look they are trying to produce. The examples above are quite 
rudimentary, more complex Styles could involve: 

• on-air camera zooms from long shot to close-ups 

• on-air camera movements from one part of the set to another 



• on-air DVE moves 

• on-air switcher keyframing effects 

Additionally, there is a transition into each Style. For the video this could be a simple cut, 
or a more complex dissolve, wipe, or DVE effect. For the audio, the new Style's audio 
can simply replace the previous audio mix, or be an addition or subtraction from the 
previous mix. 

For ease of production, each Style is given a name. The show consists of the Style List. 
Where there is a script, the Styles are associated (embedded) with the script, and the Style 
List is derived (extracted) from the script or scripts. 

A Style is embodied as one or more N-MEMs. A Style Factory manufactures the Styles 
themselves, and so it is the Style Factory which creates the N-MEMs. This output of the 
Style Factory is placed in the Style Library. The data in an N-MEM object is a snapshot 
of the overall state of all the controlled devices (including video switcher, audio mixer, 
cameras and robotic pedestals, character generators, still stores, video and audio digital 
disk recorders, and VTRs). The N-MEM extends the work (previously patented and 
trademarked by Grass Valley) on developing E-MEM technology for switchers. Where 
an E-MEM (Effects Mem ory) is a snapshot of the state of a single device used to produce 
a desired artistic effect on the screen, the N-MEM is a snapshot of the overall state of all 
the controlled devices, used to produce an overall artistic effect. An N-MEM is a set of 
E-MEMs. When an E-MEM is learned, the snapshot of the device is stored for later use, 
when an E-MEM is recalled, the saved snapshot of state is used to bring the device back 
to the state when the snapshot was originally taken (learned). . 

Creation of these E-MEMs is one of the functions of the Style Factory. However the 
creation of an N-MEM is much more than the creation of a single snapshot of multiple 
machines (which would just be a super-E-MEM). The N-MEM contains the superset of 
E-MEMs which reflects all possibilities for realization of the Stvle . 

To understand this let us consider a simple example of a Style which we shall call "close- 
up with title". On screen would be a close-up of the on-air talent, with a title 
superimposed, plus audio from the performers microphone. To realize this example 
effect, the control system has to recall E-MEMs which at the very least: 

• recalls the position of the camera (using robotic pedestal and/or pan-tilt head) • 

• recall the camera GCU settings and lens settings 

• recall the camera source on the switcher background bus 

• recall the CG page 

• . recall the CG source on a switcher Key bus 

• recall the audio slider to the setting for the microphone 

For the above example, the Style would have one N-MEM. Considering just part of this 
N-MEM, it could have an E-MEM for each of the possible performers (e.g. Talent-1 
(John), Talent-2 (Fred) , and Talent-3 (Jane3)). In turn, each of the these anchors could 
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be taken with either Camera- 1, Camera-2, or Camera-3. Thus the E^MEMs forming the 
N-MEM contain the whole range of legitimate pre-learned possibilities. 

To summarize and recapitulate, the job of the Style Factory is to include all the possible 
E-MEMs that could be used to realize a Style into am N-MEMs which is placed in the 
Style Library. 
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STYLE FACTORY 

The Producer or creative personnel planning a television program need to know nothing 
about E-MEMs or N-MEMs.. They concentrate on associating Styles with their stories 
and scripts. Where there may be a bewildering array of possible E-MEMs and E-MEM 
combinations in a Style, there is a relatively small set of Styles used in a particular show, 
generally less than 20. The creative staff also makes selections such as who is on-air (i.e. 
who is reading the script), and the particular text for a title or other graphic. This begins 
the process of N-MEM parameterization, which is described below. 

It is a program's Director or other more technical personnel who complete the process of 
N-MEM parameterization. Parameterization involves the selection of the specific E- 
MEMs in the N-MEM to be used in the particular use of the Style. It is these 
parameterized N-MEMs which are placed into the Effects List. Such an N-MEM is called 
a parameterized N-MEM to distinguish it from a library N-MEM. 

Parameterization is made easy by not dealing with abstract entities such as E-MEM 
numbers, but by dealing with named resources which are all in the system resource map. 
For example, when the Producer selects "Jane" as the performer, the list of camera E- 
MEMs is constrained to a smaller set. When the Director selects camera-2 for the style to 
shoot Jane, then the parameterization is complete and the camera E-MEM has been 
selected. Also, when the switcher E-MEM is recalled, because the camera has been 
selected, the correct input is selected on its background bus. Other examples of 
parameterization are, placing text such as "Live from LA" as the title for a CG, selecting 
which clip to play from a video server and on which output channel, which VTR 
machine to use for a specific tape, and which remote feed channel a live microwave 
transmission will come from, etc. 

Once parameterization is complete, the effects list is ready to go on-air. One element of 
the described invention is a rules checker, which in the background checks that each story 
element has an associated Style, and that each Style has it's N-MEMs fully 
parameterized. Hours before the show starts, the rules checked will indicate much is 
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incomplete. Ideally, when the show starts, there should be no problems indicated. The 
rules checker also checks for resource conflicts, and local station policies. 

When the Effects List is ready to go on-air it is passed on to Run Control. Play-out of 
the event list is a manual operation. At its most basic, the user presses the 'Take' button. 
The current N-MEM is released, the next N-MEM goes on air, and the N-MEM after that 
is previewed. Before an N-MEM goes on air, it should be in the ready state - and the N- 
MEM is ready when all. the controlled devices are cued. There is an E-MEM for each 
controlled device and each E-MEM is cycled through the states: idle, cueing, 
ready(cued), active (on air or on preview), and released. The N-MEM's state is a rollup 
of the E-MEM states, and it cycles through the states: idle, cueing, ready(cued), on- 
preview, on-air, and released. This is all described in much more detail in the (draft) 
Product Technical Specification. 
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One aspect of importance is to note that although what goes on air is manually paced by 
pressing *Take\ the cueing of devices is performed autpmatically. This is different from 



other systems which require the cueing of a device (such as a camera move) to be 
manually pre-programmed onto a timeline. When an E-MEM is released, Tsunami scans 
forward through the event list for the next N-MEM which has an E-MEM for the same 
device, and brings this into the cueing state. So for example, as soon as the camera E- 
MEM is released from its last usage, the next E-MEM in the Effects List that uses this 
camera is found, and that E-MEM is cued, to bring the camera into its next position, 
ready for going on air. 

« 

Besides a simple 4 take' , Run Control supports operations which 'float' (skip) an N- 
MEM, 'take audio 9 to bring on the audio in advance of video, to 'hold audio' after the N- 
MEM is released, to 'roll' tapes and DDRs, and more. All this too is described in the 
Product Technical Spec. 

One further consideration is the need to 'tweak' the controlled devices after their E- 
MEMs have been recalled. For example, the camera might have been moved to the 
correct position, but the performer is slightly out of position; in this case the camera 
needs to be moved. Another example would be the perfonrier talking more quietly than 
expected and the gain on his microphone needing to be adjusted. Although this tweaking 
can be performed 'blind', it is best done when the video or audio is on preview, or even 
on-air. Control surfaces (GUIs and physical control panels) are provided for making such 
adjustments, which are stored as deltas to the E-MEM. What is significant is that the 
operator has the choice of having this delta automatically applied each time the resource 
is used in an E-MEM. So if the "talent-left" Style had to have the camera position and 
focus adjusted slightly, each time the camera goes to the same position, the same delta is 
applied. Similarly for the use of a microphone. 

■ * . 

This concludes the description of Styles as used in ordered (scripted) shows. For 
unscripted shows, or shows with unscripted sections, the Style Box is used. The Style 
Box consists of buttons (on a GUI or physical control panel) each of which is associated 
with a Style. Each Style is parameterized when it is placed in the Style Box. Run Control 
operations associated with these styles are to: 

• cue the N-MEM - making it ready for air 

• making the N-MEM next - so it appears on video and audio preview 

• making the N-MEM current - so it is taken to air 

Combination operations 'cue and next' , 'cue and current (on-air)* are also supported. 

■ 

The purpose of the Style Box is to allow the operator to quickly prepare a style and take it 
to air. A typical system would have a Style Box with 20 buttons, with a total of 10 pages 
of styles capable of being placed onto these buttons, making 200 Styles containing 
parameterized N-MEMs. However, to be used effectively the 20 exposed Styles under the 
operator's finger tips should be the ones most likely to be used on an immediate basis as 
the live show progresses. Other operations used to support the Style Box are: 

• bringing a Library Style into the Style Box and parameterizing it 

• re-parameterization on-the-fly of a Style in the Style Box 

• moving Styles from one bank to another in the Style Box 
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Now that the basic inventions are described, this document shows how it is used for an 
Integrated News Production System, named Tsunami. The described inventions are 
embodied as a control system running on a computer. In common with existing news 
production systems, this control system (Tsunami) receives a 'running order' for the 
show from a NRCS (such as iNews and ENPS). What is unique about this system is the 
way it works. Tsunami perceives received running orders as Style Lists and so converts 
these running-order/style-lists into an Effects lists, i.e. ordered sequences of N-MEMs. 
Tsunami does not employ timelines, where the items in the timeline have durations, nor 
does it employ macros, which are stored (learned) sequences of commands to be sent to 
the connected devices. 

In the workflow of producing a news program, stories or slugs are entered into the 
rundown, and placed in the order they will be shown. The producer associates a Tsunami 
Style with each story in this rundown by employing the Tsunami software ActiveX 
Plug-in (TAP) to the newsroom computer system. The TAP communicates with the 
Tsunami Controller to retrieve a list of the contents of the Style Library and offers the 
choice of Styles to the Producer. The TAP is also active in extracting information from 
the scripts and rundown which describes which news anchor is reading which story, and 
text for graphics. This information is used to start parameterization of the N-MEMs as 
already described. 

The running order of scripts (i.e. the Style List) is transferred to Tsunami via variety of 
means including FTP and MOS protocols. The Director (and/or Producer) can proceed to 
parameterize the N-MEMs either using the TAP on the NRCS or using Tsunami's own 
GUI. As described, the rules checker ensures everything is correctly parameterized. 
Tsunami generates an Event List for each rundown (i.e. for each news program). 

Tsunami's GUI shows the event list, the state of each N-MEM, and the state of the E- 
MEMs. Once Run Control is activated, the GUI indicates progress through the Effects 
List. Additionally the GUI highlights any anomalous conditions which require manual 
intervention, such as an N-MEM not being cued because the source material is not 
available or the physical device has failed. In this case the operator could skip over (float) 
the story, and optionally come back to it later when the source material becomes 
available. 

Another feature unique to Tsunami is that unlike other Integrated News Production 
Systems, Tsunami accepts new running orders and modifications to existing running 
orders at all times from the NRCS, even when the show is on-air. It is capable of doing 
this, because the scripts forming the running orders are received with embedded 
references to Tsunami Styles. Further, any errors or omissions in parameterization of the 
N-MEMs within these Styles, are immediately flagged by the Rules Checker. 

One last point to note, is that not all the devices controlled by Tsunami have their own 
internal E-MEM mechanism. Some devices do, others do not. Where the device has its 
own E-MEM mechanism, Tsunami's E-MEMs are references to the E-MEM information 



stored in the device. Where there is no E-MEM mechanism in the device, Tsunami acts 
as a proxy for the device and stores the information locally when the E-MEM is learned, 
and then sends the state information back to the device when the E-MEM is recalled. 



